Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ignore blocked datasets in WorkerSize metrics for auto scaling #2949

Merged
merged 4 commits into from
Jun 26, 2024
Merged

Ignore blocked datasets in WorkerSize metrics for auto scaling #2949

merged 4 commits into from
Jun 26, 2024

Conversation

AndreaFrancis
Copy link
Contributor

Fix for #2945

@AndreaFrancis AndreaFrancis marked this pull request as ready for review June 25, 2024 17:54
@AndreaFrancis AndreaFrancis requested a review from a team June 25, 2024 22:02
@AndreaFrancis AndreaFrancis merged commit 21cea15 into main Jun 26, 2024
25 checks passed
@AndreaFrancis AndreaFrancis deleted the ignore-blocked-in-worker-size-metrics branch June 26, 2024 15:15
@severo
Copy link
Collaborator

severo commented Jun 26, 2024

I think we forgot to decrease the metrics when a dataset is blocked:

dataset-viewer/libs/libcommon/src/libcommon/queue/dataset_blockages.py

Line 75 in ae00789

DatasetBlockageDocument(dataset=dataset, blocked_at=get_datetime()).save()

On the other side, when the dataset is not blocked anymore due to the TTL index, we will have to wait for the next metrics update cron job to fix the metrics

@AndreaFrancis AndreaFrancis mentioned this pull request Jun 26, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@severo severo severo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@AndreaFrancis @severo